Consider learning a policy from example expert behavior, without interaction with the expert or access to a reinforcement signal. One approach is to recover the expert's cost function with inverse reinforcement learning, then extract a policy from that cost function with reinforcement learning. This approach is indirect and can be slow. We propose a new general framework for directly extracting a policy from data as if it were obtained by reinforcement learning following inverse reinforcement learning. We show that a certain instantiation of our framework draws an analogy between imitation learning and generative adversarial networks, from which we derive a model-free imitation learning algorithm that obtains significant performance gains over existing model-free methods in imitating complex behaviors in large, high-dimensional environments.

In appendix, we Reward OptimizationWepropose .

We further discuss the sample complexity for a desired approximation accuracy.

In several problems of interest, it is possible to observe the behavior of multiple agents with different degrees of expertise .

Neural Information Processing Systems http://nips.cc/

Temporal logics have been used in prior research as a language forexpressing desirable system behaviors, and canimprovetheinterpretability ofspecifications if expressed as compositions of simpler templates (akin to those described by Dwyer et al. [2]).

We formally guarantee that the distributed learners asymptotically achieve consensus which belongs to the set of stationary points of the bi-level optimization problem.